6.1 Error Correcting Codes – Preliminaries

In today's lecture, we will discuss the application of expander graphs to error-correcting codes. More specifically, we will describe the construction of linear-time decodable expander codes due to Sipser and Spielman. We begin with some preliminaries on error-correcting codes. (Several of the proofs presented in this lecture are adapted from the lecture notes of Venkatesan Guruswami's course on Codes and Pseudo-random objects [Gur1]). We first recall the definition of error-correcting codes. For more information, see, e.g., the excellent survey by Guruswami [Gur2]. Suppose Alice wants to send a k-bit message to Bob over a noisy channel (i.e., the channel flips some bits of the message). In order for Bob to recover (decode) the correct message even after the channel corrupts the transmitted word, Alice instead of sending the k-bit message, encodes the message by adding several redundancy bits and instead sends an n-bit encoding of it across the channel. The encoding is chosen in such a way that a decoding algorithm exists to recover the message from a codeword that has not been corrupted too badly by the channel. (What this means depends on the specific application.) More formally, a code C is specified by a injective map E : Σ k → Σ n that maps k-symbol messages to n-symbol codewords where Σ is the underlying set of symbols called the alphabet. For the most most of today's lecture, we will only consider the binary alphabet (i.e., Σ = {0, 1}). The map E is called the encoding. The image of E is the set of codewords of the code C. Some times, we abuse notation and refer to the set of codewords {E(x)|x ∈ {0, 1} k } as the code. k is refered to as the message-length of the code C while n is called the block-length. The rate of the code, (denoted by r), is the ratio of the logarithm of number of codewords to the block-length n, i.e, r(C) = log(#codewords)/n = k/n ≤ 1. Informally, a rate is the amount of information (about the message) contained in each bit of the codeword. The (Hamming) distance ∆(x, y) between any two strings x, y ∈ {0, 1} n is the number of bits in which they differ. The distance of the code, denoted by d, is the minimum Hamming distance of any two of its codewords, i.e., d(C) = min x,y∈C ∆(x, y). The relative …